Miminum Entropy Set Cover Problem for Lossy Data Compression

Classical minimum entropy set cover problem relies on the finding the most likely assignment between the set of observations and the given set of their types. The solution is described by such partition of data space which minimizes the entropy of the distribution of types. The problem finds a natural application in the machine learning, clustering and data classification. In this paper we show that it is closely related to lossy data compression. In particular, we prove that the minimum entropy set cover is a special case of specific generalized entropy coding. We establish the relation between the solution of these two problems. Moreover, we propose a simple greedy algorithm which approximates the entropy of our lossy data compression within an additive term of log2 e. The proof is based on the recent result obtained for minimum entropy set cover and our partition reduction theorem for lossy data compression.